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ABSTRACT 

In 1984, suaaer bridge students in the Indiana 
University Learning Skills Center were asked to coaplete an 
open-ended questionnaire on PLATO coaputer assisted instruction (CAI) 
used in their study skills courses. Of 226 students responding to the 
survey, 175 (77%) had chosen to use PLATO; 86 students used both the 
psychology and sociology review lessons, while 49 used only the 
foraer and 35 only the latter. The remaining students used only 
lessons which were not directly related to the study skills program, 
e.g., aath or English language lessons. Students used PLATO for a 
variety of reasons, i.e., convinced by instructors, attracted to the 
coaputer format, persuaded by friends, reduction of study tiae, 
better understanding of course aaterials, or to gain experience on a 
coaputer. Although trained neither in coaputer science nor in 
evaluation of education prograas, the students identified each of 
four requisites for success identified in literature discussing CAI 
effectiveness: (1) implementation aust be voluntary; (2) it must be 
part of a larger, overall educational prograa and inpleaented in 
conjunction with larger educational objectives; (3) the software aust 
take advantage of the coaputer* s interactive capabilities and it aust 
be of pedagogical quality; and (4) the student aust not be in primary 
control of the pacing. It is suggested that students have a wealth of 
knowledge which educators could and should tap in order to vastly 
iaprove the effectiveness of educational prograas using coaputers. k 
list of references and copy of the student questionnaire are 
provided. (JB) 
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ABSTRACT 

Articles discussing the effectiveness of computer-assisted 
instruction have all too often ignored the perceptions of the 
students who use the software. Student input has long been an 
effective means of improving teaching and of evaluating instruc- 
tional materials. It seems reasonable that student input could 
similarly be used to evaluate and improve CAI. 

In 1984 summer bridge students in the Indiana University 
Learning Skills Center were asked to complete an open-ended 
questionnaire on PLATO-CAI used in their study skills course. 
Although trained neither in computer science nor in evaluation 
of educational programs, the students identified each of four 
requisites for success which had been identified in literature 
discussing CAI effectiveness. The implication is that students 
have a wealth of knowledge which educators could and should tap 
in order to vastly improve the effectiveness of educational 
programs using computers. 



STUDENTS ARE CAPABLE OF ASSESSING THE EFFECTIVENESS 
OF COMPUTER-ASSISTED INSTRUCTION 

Through tremendous advances in software, particularly 
through new authoring systems, mainstream teachers are 
beginning to produce educational software for their students' 
use. Although the medium is new, teachers have traditionally 
produced their own handouts and other supplementary educa- 
tional materials. Teacher-produced software may be geared 
more towards individualized instruction than traditional 
handouts, but other than that its general purpose remains 
the same. 

Once teachers are actively producing CAI and students are 
using it, they must necessarily determine its ef tectiveness. 
The problem is that in assessing CAI there has long been a 
lack of consensus as to what constitutes effectiveness: 

The effectiveness of CAI has been defined differ- 
ently by different investigators. To some, 
effectiveness means the amount of learning that 
takes place initially. To others it means the 
degree of retention of learning, or at the very 
least, whether or not an individual stays in or 
drops out of a learning experience. Still others 
are concerned with the learner's change in attitude 
toward the computer as an instructional medium or 
simply as a helpful tool in the culture. Finally, 
owing to the fact that CAI is in its infancy, some 
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are simply concerned with transportability of mate- 
rials and/or acceptance of the materials for use by 
others (Walker & Hess, 1984, p. 3). 
Just as they traditionally provide guidance in evaluating 
teaching and instructional materials, teacher education 
programs will at some point provide guidance in determining 
the effectiveness of CAI. This task should not be nearly as 
complicated as the computer journals would have us believe. 
Software is traditional supplementary material in a new med- 
ium, so the basic precepts of its evaluation should remain 
unchanged. 

Since computer technology provides a range of interactive 
features not found in traditional media, and since software is 
considerably more expensive than traditional media, teachers 
should not simply be satisfied with effectiveness. If it 
cannot be shown that CAI is a more effective means of learn- 
ing, the technology has not been fully tapped and money has 
been wasted. For these reasons the Indiana University 
Learning Skills Center has developed appropriate software and 
implemented CAI in a manner which appears to be consistent 
with this tougher standard. 

By comparing expected and actual scores on psychology and 
sociology exams, for example, we found that some 200 high risk 
freshman received statistically significant higher scores in 
the summer of 1982 because they had chosen to review for the 
test using PLATO-CAI lessons (Woelffer and Chase, 1983). 
The lessons had been specifically prepared for use in Indiana 
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University's summer bridge program. A similar study using 
data from the 1984 summer course supported the original find- 
ings (Luk and Jacobs, 1985). The Learning Skills Center staff 
was encouraged by the results of this study, but we remained 
somewhat skeptical in light of other published reports of less 
favorable results. 

We therefore turned our attention to variables which 
might explain CAI's success in some published studies and its 
lack of success in others. After researching the literature 
on this topic, we concluded that there are several requisites 
for successful CAI: (1) it must be implemented on a voluntary 
basis, (2) it must be a part of a larger overall educational 
program, and implemented in conjunction with larger educa- 
tional objectives, (3) the software must take advantage of the 
computer's interactive capabilities and it must be of peda- 
gogical quality, and (4) the student must not be in primary 
control of the pacing (Hartig, 1984). Although the literature 
presented valid arguments which led to these conclusions, 
we were puzzled by how little attention the researchers paid 
to the perceptions of the students who were being studied. 

In most cases the studies were limited to objective 
measures of performance. Sometimes, but not always, they 
included a student attitude survey. In a few studies student 
input was more formally solicited, but the resulting informa- 
tion was of little value because the requested information was 
very basic. We are capable of learning much more about CAI 
from our students than simply how many students had used 
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computers before, how many liked a particular CAI lesson, and 
how many would lik x to do further study on the computer. 

In his study of Journalism CAI at Indiana University, 
William Oates in 1982 was one of the first to use student 
input in a more substantive manner. He found that instructors 
consistently responded more favorably to CAI materials than 
students. Other researchers have since substantiated this 
finding. The instructor/student discrepancy in fact seems to 
hold true for both instructional and motivational values: 

The reported differences between teacher and student 
evaluations can be attributed to their different 
roles. Teachers, as content specialists, are more 
critical of the specific content of a program. 
Knowledgeable in teaching strategies and the 
development of the concepts, they are aware of the 
varying pedagogical complexities for the different 
topics. Students, not cognizant of these intrica- 
cies, evaluate programs on the basis of interest, 
clarity and their level of participation (Signer, 
1983, p. 35). 

Of fir (1983, p. 28) found that, in regard to educational aims 
in general and CAI in particular, "...the actual situation in 
reality is closer to the opinions of the students." If this 
assertion is true, future research into CAI effectiveness must 
investigate the great wealth of knowledge held by our stu- 
dents, the true CAI experts. 
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In the summer of 1984 T wanted to get a clearer picture 
of the students 1 potential in software evaluation and design, 
so I gave our high risk freshmen a rather lengthy question- 
naire (see Appendix). These are students who did not meet 
Indiana University's admissions criterea, but who were 
admitted on a special summer bridge program upon the recom- 
mendation of high school counselors throughout the state. One 
requirement of the summer program is that the students enroll 
in a study skills course which in small groups teaches tech- 
niques for studying in two large lecture classes: intro- 
ductory psychology and introductory sociology. Despite the 
fact these students were not well prepared academically and 
that they at times lacked motivation , their answers to the 
questionnaire provide support for Offir's position. 

Of 226 students responding to the survey, 175, or 77% had 
chosen to use PLATO. Eighty-six students used both the 
psychology and sociology review lessons, while 49 used only 
the former and 35 only the latter. The remaining students 
used only lessons which were not directly related to the study 
skills program, e.g., math or English language lessons. 
Students used PLATO for a variety of reasons: 

78 students were convinced by instructors of 
PLATO'S value, 

70 were attracted to the computer format 
and technology, 

68 students were persuaded by students who had 
previously used PLATO, finding it valuable, 
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56 thought PLATO could reduce their total 
study time, 

42 felt PLATO would lead to better understanding 

of the course material, 
32 thought PLATO would enable them to do better 
on the tests, and 
4 students said simply that they wanted 
experience on a computer. 
Many students of course checked or wrote in more than one 
answer to this question (#2 on the survey), thereby confirming 
that the decision to use or not use CAI can be fairly complex. 

The responses to question #7, answered by students who 
did not use PLATO, were equally varied. These answers, 
however, fit rather conveniently into three broad categories: 
(1) students didn't know about PLATO, (2) they intended to us~ 
PLATO but never got around to it, or (3) they felt, some 
correctly and others not, that they were doing OK without it 
and so preferred not to use it. 

As previously mentioned, CAI in the Learning Skills 
Center has been predominantly voluntary in nature. Research 
into personality types has shown that certain people simply do 
not mesh well with computers (Hoffman and Waters, 1982). It 
would be counterproductive, then, to force these people to use 
CAI. Without knowledge of this research, our students made 
very similar comments t 

I used PLATO because I was curious about 
computers. People who aren't curious about 
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computers shouldn't be required to use it. 

PLATO holds my attention because of my interest 
in computers. Other students might not feel the 
same way. 

I don't like computers. I flunked computers in 
high school. I don't work well with computers, so I 
chose not to use PLATO. 

I had a frightening experience with computers 
before, so friends had to work hard to convince me 
to use it. 

On this issue the students are overwhelmingly pro-choice, and 
in some instances negative feelings would obviously have 
resulted from mandatory CAI. Not a single student commented 
that PLATO should have been required. 

The aforementioned comments car- in response to question 
#4 on the survey. The appropriateness of CAI as an individual 
decision is underscored by the objective portion of this 
question: 

65% felt that learning from PLATO was more effective 
than learning the same materials from books 
and notes, 

30% felt that learning through the vaiious media 
was "about the same," and 
5% felt that PLATO was a less effective means 
of learning. 

The problem for educators is that, depending on the particular 
student in question, each of these responses may be equally valid. 
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Just as CAI is best left voluntary, it is also more 
successful when it is an integral part of a larger educational 
program (Camstra, van Dijk, van der Avoird, 1979). In other 
words, CAI cannot stand alone. It must be implemented with a 
specific learning objective in mind in order to contribute to 
a larger learning objective. In this vein, t^o, our students 
made some perceptive comments: 

Neither CAI nor books can be effective without 
the other. 

Books can be used without PLATO, but PLATO not 
without books. 

Sometimes the textbook and discussion in class 
was difficult to understand, and PLATO helped a lot. 
PLATO can't stand alone, though. 

PLATO covered the most important material, but 
it didn't cover everything that was important. 

I did OK in class, but only because I relied 
more on my notes than on PLATO. 
The overall goal of the summer program was to prepare students 
for the coursework they would complete in the coming fall 
semester and beyond. The students obviously recognized that 
various components of the overall program worked together to 
achieve this goal. PLATO was simply one of those components. 

The third requisite for success, that CAI must be inter- 
active and of pedagogical quality, seems quite obvious. It is 
perhaps a reflection on an "educational software" industry 
which seldom mploys educators that much of today's commercial 
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software cannot be described in these terms. Formal support 
for interactive CAI has come from various research efforts 
(e.g., Avner, Moore and Smith, 1980; and Schloss, Schloss and 
Cartwright, 1984). I anticipated comments from our students 
in support of interaction, and the students did not let 
me down: 

PLATO is more effective because books and notes 
are passive. 

PLATO saved me study time. To do the lessons 
I had to get more involved with the material. I got 
more involved with studying this term because I got 
into it. 

PLATO gave a better, interactive relationship 
to work in. 

I used PLATO before in another class and knew 
it could be a better way to learn because it 
requires you to get actively involved with material. 

PLATO could be more effective if it 

— had more content, more examples and 
more questions 

— were capable of handling essay answers 
generated the terms randomly 

gave us the choice of more varied types of 
things to do. 

Interactive software, however, must not proceed at the 
expense of pedagogical quality (Fisher, 1982). Our students 
were quick to point this out, too. At least a dozen students 
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indicated that *he psychology and sociology material was 
presented more clearly on PLATO than in their books and .n 
class, and that it was primarily for this reason that ?LAT0 
was a more effective way to learn. In addition, sev ..*a 
students mentioned that they had previously been exj sed o 
poor quality software and as a resuiw hadn't learn* thing. 

The final requisite for successful implemer m of CAI 
is the c pacing must be externally controlled (S.eiiberg, 
1977). This would appear to be a mor difficult concept for 
persons with little training in computer science. As might be 
expected, only one of our 175 PLATO ubt. " broache 1 this topics 
It was good because PLATC ^ me through the 

exercises step-by-step and wouldn * let me go on 

before I was *eady. 
With this single comment, though, our students as a group had 
made appropriate mention of each ingredient for successful CAI. 

College teachers have long solicited student input at the 
end of each term on both classroom instruction and instruc- 
tional materials. While some teachers make little use of 
their students 9 evaluations, others know that -his data can be 
valuable in improving their courses in the future. The 
Learning Skills Center questionnaire indicates that this same 
appro:.:h, if the right questions are asked, can provide a 
tremendous amount of information which can potentially improve 
both software design and the use of CAI in our schools. The 
students 1 words will not be as eloquent as those of the 
researchers, and students are not in a position to support 
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their claims with measurable data. The important point is 
that our students know a great deal about the appropriateness 
of instructional tools and how they are used. Instead of 
assuming that we have little to learn from students about CAI, 
educators need to take steps to tap the students' knowledge in 
this area. 

In addition to demonstrating the students' expertise in 
the area of computer-assisted instruction, the 1984 question- 
naire showed that students for the most part held PLATO-CAI in 
high esteem. The responses to question #3 provide perhaps the 
best support for this assertion. Of 144 respondents, 31% 
claimed to have spent more time studying because of PLATO. 
This response was favorable in all but one instance. In 
open-ended comments students explained that they were more 
interested in the material because of PLATO, and so spent more 
time learning. The 55% who spent less time studying because 
of PLATO also made numerous positive comments. They learned 
more easily because FLATO explained more clearly and r^ovided 
wetter summaries. Thirteen percent of the students felt the 
amount of study was about the same as it would have been 
without PLATO, and the remaining 3% werp unsure of any effect 
PLATO may have had on study time. 

The overall favorable reaction to PLATO was consistent 
with our own observation of students during the summer, as 
well as with the previously mentioned statistical data on test 
scores. In certain other ways, however, the questionnaire was 
more a cause for concern than for rejoicing. Despite lengthy 
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periods of "down time" and numerous reports of terminal 
malfunction over the course of the summer, an unlikely 94% of 
the respondents reported that PLATO worked properly when they 
used it. Only 5% reported that it did not, either part or all 
of the time. Also despite numerous complaints during the 
summer, IU's PLATO facilities got a 91% rating of approval. 
In retrospect it seems the students had a more positive 
reaction to the facilities than did the instructors. When the 
students 9 comments were negative, they generally cited the 
need for additional terminals. 

Even more disturbing, particularly in regard to research, 
is that at least 15% of the students signed on at one point or 
another under assumed names. I had learned from several 
instructors that some students had worked with friends at the 
PLATO terminals. This activity o.n be a productive learning 
experience, but it possibly means that some 25 additional 
students with lower- than -expected test scores did use PLA10, 
or vice versa. The explanation most often given for using a 
friend's s~gn on was a forgotten password. 

It is admittedly not easy to tally the results of 
open-ended student surveys. It takes considerable time to 
decipher handwriting and to make sense out of comments 
It is also more likely that teachers and administrators will 
obtain disconcerting results than it is with traditional 
evaluation forms which are primarily objective in nature. 
With these caveats in mind, I would very much encourage 
educators to use this approach when assessing the role of CAI 
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in their curricula. Little is known about the proper use of 
CAI in any specific subject area. Open-ended questionnaires, 
despite their drawbacks, can provide considerable useful 
information. 



ERIC 16 



— ^ ^ ~ * --^^ j - - _ t .H**.. 



REFERENCES 

Avner, A., Moore, C. and Smith, S. Active external control: 

superiority of CBE. Journal of Computer- Based Instruction , 
1980, 6, 115-118. 

Camstra, B., van Dijk, T., and van der Avoird, W. Leren met 
de computer , eindrapport van het PLATO-p roof project , 
Amsterdam? University of Amsterdam, 1979. (ERIC 
Document Reproduction Service No. ED 194 104.) 

Cornick, L.B. Foreign -language CAIt A checklist. 
Un terrichtspraxis , 1984, 17, 315-318. 

Fisher, F.D. Computer-assisted education: "What's not 
happening?" Journal of Computer-Based Instruction , 

1982, 9, 19-27. 

Fisher, G. Where CAI is effective: A summary of the research. 

Electronic Learning , 1983, 3 (3), 82-84. 
Hartig, G. Implementing CAI in a university learning center. 

Journal of Computer -Based Instruction , 1984, 11, 

113-116. 

Hathaway, M.D. Variables of computer screen display and 
how they affect learning. Laucational Technology , 
1984, 24 (1), 7-11. 

Herbert, M. You can do it! Modify software for your 
students' needs. Journal of Business Education , 

1983, 58, 178-180. 

Hoffman, J.L., and Waters, K. Some effects of student 
personality on success with computer-assisted 
instruction. Educational Technology , 1982, 22 (3), 
20-21. 



15 



Hord, E.V. Guidelines for designing computer-assisted 

instruction. Instructional Innovator . 1984, 29 (1), 
19-23. 

Krulick, J. A. Synthesis of research on computer-based 
instruction. Educational Leadership . 1983, 41 (1), 
19-21. 

Luk, H., and Jacobs, L.C. Use of PLATO by Groups students 

in sociology an d psychology (Research Report Number 22). 
Bloomington, IN: IU Bureau of Evaluative Studies and 
Testing, Wright Building of Education, June 1985. 

Oates, W.R. Effects of computer-assisted. instruction in 
writing skills on journalism students in beginning 
r.ewswriting classes. (Doctoral dissertation, Indiana 
University, 1982). Dissertation Abstracts International . 

1982, 43, 2822A. (University Microfilms No. DA 
8301116, 312). 

Offir, B. Attitudes of university instructors and students 
toward using computers for learning: Discrepancies 
between thought and action. Educational Technology , 

1983, 23 (5), 26-28. 

Schloss, P.J., Schloss, C.N,, and Cartwright, P. Efficacy 
of four ratios of questions and highlights to text 
in computer-assisted instruction modules. Journal of 
Computer-Based Instruction . 1984, U, 103-106. 

Signer, B. How do teacher and student evaluations of CAI 
software compare? The Computing Teacher . 1983, 11, 
34-36. 



ERIC 



IS 



16 



Steinberg, E.R., Review of student control in computer-assisted 
instruction. Journal of Computer- Based Instruction , 
1977, 3, 84-90. 

Van Handle, D.C. CAI in an intensive German class, 
Unterrichtspraxis , 1984, 1/7, 312-314. 

Woelffer, K. , and Chase, C.I, Use of PLATO by Groups 

students in sociology and psychology (Research Report 
Number 21). Bloomington, IN: IU Bureau of Evaluative 
Studies and Testing, Wright Building of Education, 
September 1983. 



ERLC 



19 



APPENDIX 



9 



20 



18 



ERIC 



CWUF8 1984 CMPVni-ASSXSTED 
IHSTRUCTIOH QUESTIONNAIRE 

1. Did you use PLATO lessons this sunmtr in your XI 52 course? 

y«s _____ no 

If no, skip to question #7. 

If so, which lessons did you use? 

______ Psych Review 

______ Soc Review 

______ Other (Plesse specify: 



Did PLATO work properly for you when you used it? 
— — _____ "o ______ unsure 

Did you use your own personel sign-on, or s friend's sign-on? 

ay own 

______ s friend's 

2. Jhy did you use PLATO? (Check as many as appropriate.) 

past students reported PLATO to be valuable for learning XI 52 outer ial 

«y instructor convinced me to use it 

I »•» sttracted to the conputer format 

1 thought it would reduce the amount of tint. I toent in reading and 

homework for XI 52 



other (please specify): 



3. On the whole, would you say you spent more time or ieM time than you 
otherwise would have spent learning the X152 material since you did it 
on PLATO? Explain your answer: 
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. Check the ltta which aost strong ly ref ltcts your opinion. 

learning on PLATO Is More effective then leernlng the same material 

from books and notes 

leernlng on PLATO Is about the same as learning the same material 

from books and notes 

learning from PLATO Is not as effective as learning the same material 

from books and notes 

Comments: 



. How can the PLATO lessons be Improved? Include specific suggestions 
for individual PLATO lessons If you wish. 



6. Are the PLATO facilities (LSC, library, Education Bldg.) adequate for 
your purposes? 



P«« no (pleese explain: 



7. If you did not use PLATO, or used it some but not ouch, please brie'li 
describe why you chose not to do so. 
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